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most ^fectiv^^^data collection ^method by exara^ining the rate and 
nature\of responses and' the cost of three meth6ds — personal 
interviVwr mailed guestionnairer. and telephone' survey* The. study 
disagree's with previous "research indicating - that the' mailed 
questionnaire is the most appropriate method. A .compar.atiye analysis 
of the three jti^thods of <:onducting a fbllow-up Study of former high ^ 
school stuae^i^fs.at the Skyline. Career Development Center (CDC)^ was 
designed tp t^ompil^ and design: survey instruments^ to incorporate' and 
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rate at less.^Qd^Sjt for both' small and large districts. For best;. ^ 
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tike student pc[g,iflation r careful use cf clerical personnel^ and 
goal^orient-ed '^ILnstrumentatiqn. (JB) • o 
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rr\ . The' practice qi conducting a followvup survey of . graduates is neither 

Qv.'' new nor innovative. Local school districts, governmental agencies, and 

O ' ■ ^ I ' . ' " • ' ^ - : , - 

«^ ^ independent researchers -have been involved 'iri such studies for over foxj^ ^ 

[3 ' ' • i ' * ^ - • * ^ / ' 

decades. Recently, 'there has been a renewal df interest in the use of follow- 

jup surveys. The increasing .emphasis on accountability imposed on educators-' 

V legislators and the general public may be an important impetus for more 

research and evaluation activities in which a follow-up survey is used, 

* Fbllow-up studies become an afterr-the-fact needs assessment based on t)ie 

* evaluation of existing programs by former students.^ 'Data of this type can 
. only be cofi^cted -through .a follow-up -survey . Such dat^ are indilfflnsible 

'if one assumes that former clients are. worthy of cotlsideratiofi in such 
matters. Therefore, the utility 6f a follow-up survey can be dichotomized 

* - * . • i ' - 

into (a) responding to pressures for accountability with factual djata and 
Cb)^ulfilling the practitioneris constant, need for data through which, he can 
' implement program improvements'. In fact, the State of Texas now requires a 
five-year fpllbw-,up on all graduates, of vocational programs funded by the ^ 

• - ' State. ^ . r • 1 ' 

\ . ' ^ ' ' - . -/^^ ' 

With the understanding 'that such studies haVe a place in research and 

evaluation, it is lunfoi^tunate that follow-up studies are typically conceived 

• t. ■ . as a cursory examination of what graduate's are doing one to five years. after 



^ -Paper presented at the annual meeting of the American Educational Research 
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leaving school. Such studias are burdensome, costly, and ^frequently are 
severely limited by poor response rates* While a great deal of research ha^s 
been conducted about the methodology and cosTtis involved in follow-up studies, 
little,, if any, of this knowledge. has made its way to local school districts. 
Hence, application, lags far behind research,- 

• It woi^ld. appeal that the educational community needs research about 
foliow-up t|echniques\ that is generaliza^ble and relevant to its setting. ^ 
This need can best be\raet'by comparing the results' of alternative methods of 
data colleption on a diverse student population. An examination of rate of 
response,' ^cost, and na|:ure of response would allow school districts/ "(regard- 
less of size or studenti composition) to select the method most apptopriate _ 

for their purposes. ^ • ! 

* . ' ' ' ' ^ ' i, ' ^ 

One other aspect— kntent— must be discussed. Historically, follow-up ' 
has involved only countf^ng heads and identifying the activities in which 
former students ar^ presently engaged. ^ It is the belief of this .researcher 
that follow-^up^ studies should provide meaningful data that will allow -for 
program alteration; consequently , the present study was designed to accurately 
compare data obtained using three >well-known methods of collecting follow-up 
information. Each method employed — personal interview, mailed questionnaire, 
and telephone survey— utilized. existing knowledge concerning the most ad- 
•vantageous way to obtain a high response *rate. An examination of the re- 
sults and instruments should prove useful to educators planning any type of ^ 

' • 0 

follow-up survey. * ' . ' . - 

The present study is an attempt to satisfy the unmet ^eed so aptly 
cited by Jackson and Rothney (1961) in the following statement-: . ^ • • 



The merits, weaknesses, and biases of both the mailed 
questionnaire and the interview^^?iave been discussed fre- 
quently, 'but there has been litjtle research designed to ^ 
assess the comparative contributions of the two procedures ♦ 
.for follow-up purposes, . ^ ? 

While the above comment was made in 1961, it is j^ust as tinjely thirteen 

ye'ars later. In fact, it could J)e expanded to include the telephone 

survey whicfh has been the s.ubject of even less, research. ' - 

While there ar^ exceptions, follow-up studies generally continue to 

be cpnducted on a large scale with disappointingly low percentages of 

return,. Consider the following sample of studies: 

1, Four mail surveys -of East Bakersfield High School 
graduates of 1947 and 1948 yielded returns ranging from 25 
• to 39 per cent\2 ^ ^ ' , ^ 

. -2. A^state-wlde study of 14,000 Utah high school- * 1- 

graduates utilizing advertising to increase returns t)b-'. ^ 
-tained a return of 52 pej cent of -the questionnaires. /* 

3. A study of the Highline District graduates of 1970 
yierded only a 68 per cent- response after replacement of 
unattainable graduates. ■ , . ^ 

4. A study ^f the Syracuse seniors shortly before 
graduation in June, 1970, has -a^ reCum rate of only 45.7*' 
per cent,5 ^ ♦ 

5. Only 49 per cent of a random sample of Tacoma house- / 
holds responded to^a survey jdonducted by the local di^strict.^ 

The returns for some of the studies cited above may seem relatively 

zood; however, extenuating circumstances exist in those instances/ For 

example, the study in Syracuse was -conducted while **students were still in 

school. The Highline study , continually substituted respondents for those who 

were unattainable and still only obtained responses from -68 .percent, 

/ ' 

i Several .points become apparent when one considers th'e research reported 



in theWea of fo,ll^w-up surveys.. First, little is known, or at> least littlle 
has been report44," about the use of the telephone in follow-up surveys. 

' • ' ■ . \ ■ ' • ^ 

Second,^ the information presentfe^ 'in, a prior review of research indicated > 
that, for large-scale surveys, the mailed questionnaire is presently con- ^ 
sidered the most appropriate.* Finally, only a small portion of the research 
in J:he area of follow-up surveys has dealt with' comparisons of methodology 
and technique; and tHose studies appear to havei serious limitations far 
utill^zation by practitioners in lodal school disftricts. * 

• • ■ ' 

Statement* of the Problem- 

» 

Based on Vork in and with local school districts, certain assumptions 
were made concerning the -present situation with regard to follow-up re- 
search. 'Thes5 were: . ^ > 

1« Some form of follow-up study is conducted in most sizable sx:hool 
/districts. The reas^on for such studies may vary greatly from district to 
^district. ' ' sfc 

, 2. Monetary and personnel constraints often limit all- but the largest 
districts in the conduct of sophisticated follow-up research* However, 

• this does not lessen the. need for the research in smaller \districts. ' 

3. Disappointing results are rarely published or released to outsiders. 



If bne ffccep^this assumption in the light of the results previously en- 
umerated, th^Sv accentuates the problem that presently exists. 

The" pji^oSiem is that follow-up research — a useful and' accepted tool — is 
often abusedjor misused by those who could benefit greatly from such research. 
Therefore, the intent \of this study is not to devise new or innovative pro- 
cedures for carrying, out a follow--up survey but, rather, the intent is 'to 
identify .the most efficient follow-up survey method, or combination of methods 



based on cost, rate' of return, and nature of rersp.onse. Instrumentation and 
^election procedures for each method examined ar6, of cotirse, cpncomitant 
^products of any such study. However, the procedure;S ai>d '/instruments re- 

' i ' ; 

present only an application of the existing body of knowledge available to 

• - • • • • 1 - 

any researcher. ^ ^, f". - - 

For nearly four" decades, follow-up studies have been conducted by "local 
school districts ai>d yet few, if any, cHanges have takeiji place regarding 

methodology. Typically, this methodology takses th^ form of sending -"ques tion- 

' ' . o . 

naires out via bulk mail and waiting for their return. After patience 

falters under the pressure from administration, resuj.ts.are released — often 

with the omission of the percei^tage response or the assurance, that the one- 

third returned are a fair representation of the entire population. As^ 

early as. 1942, Nichols responded' to the situation thusly:, 

WVjy set great store by the results of a follow-up 
study .that got returns from but 59.9 per cent of the group ^ 
studied? Why not take cognizance of the fact that those 
who are least proud of their after-school adjustments ^re 
least likely to reply? ■ ^ 

While a 100 per cent response^ji^ate seems unlikely^ it is only reasonable to 

strive for r^su-lts in excess of 60 percerjt since §cbool administrators^ 

continue to make decisions based on returns as low as 30 per cent and rarely 

greater than 50 per cent. - ^ 

It was the purpose of this study to seek answers to the following , • 

^ questions: , 

1. Which method of conducting a follow-up study elicits the greatest 
percentage of response? . 

2. ,Are there any differences in the nature of responses elicited by 
each method of conducting a follow-i^p study? 



3. What are the comparative costs, for each method? , ^ 

4. Which method* or ^combirtation of ^nethods is most* qost-eff ective for 
conducting a large-scale follow-up study? ^ 

5. Are some methods more effective than others for certain ethnic 
groups? ^ \ • . . ' 

6. What would-be the most advantageous design for a typical follow-up 
study? * ' V 

In addition, it is hoped that the versatile set of survey instruments developed 

will have application po^ntial, with minor alterations, for any secondary or' 
vocational school follow-up^ survey pf graduates. - ^ . 

It is hoped that the above-mentioned contributions will encourage mcfre 
reliable studies at a 'known cost so that decisions can be b^sed on the best 
possible data. Another need for this study is found in the continually in- 
creasing. demands being placed upon local districts to provide follow-up data 
concerning graduaftcs and drop-outs. As mentioned%arlier , the State of Texas 
requires a follow-up study conducted over five years on all students who were 

' enrolled in ^^approved" vocational training courses. However, no detailed 
plans for .conducting the .follow-up have been provided since the issuance of 

.guidelines by many states during the 1940s and early '1950s. There' also 
exists "some question as to what information is most desirable. There 

,preset>tiy Vs a preoccupation with what former students are doing rather than 
the inclusion of questions concerning what they might recommend in the area 
:of program development* * - , 

Limitations of the Study ' • 



- Jhe most obvious s^limitation, apparent in this study, as well as other 
similar ^studies;^ is the inability to assure that inroads will be made into 



existing practices — even thp^ifh the, research is designed to produce results 
useful to public sdj^^^T districts . Another limitat±6n is that each method may 
be approached "differently and therefore the findings "reported here will ^only 
be applicable to studies similarly conducted. However, steps have been taken 
to assure that the best-inown proced.ures will be employed for each method; 

Finally, a limitation previously alluded to pertains to the instruments. 
Each instrument. While alterable^ was designed for use with the specific 
population under study. However, it Was not the intent of this study to 
provide a set of questions applicable, without changes, to any and all'^secondary 
schoon settings. ^ . # . ' 

Design of t^he Study • ^ 

^ In review of the limitations^ of the study discussed in )the, preceding 
sectioli, it was imperative that specific goals and objectives be established < 
for the study. The design of the study would then follow as a logical ex-- 
tension of these goals and objectives, incorporating proven techniques to'- 
ensure the utility and generalizability of, the findings. 

The following goals and objective^ were specified as the basis for and 

\ 

the intent *of the study: \ ^ > . : 

. . ^. ' 

Goal I. To conduct a cptbparative analysts of three different methods of 

collecting follow-up data f rom^.f ormer high school students. 

Objective 1: Identify the res^ponse rate for each of the methods 

under study. ^ \ * 

Objective 2: Assess the advantages or disadvantages of each method 

in terms of the actual responses obtained. 

Objective 3: Provide a cost-effectiveness measure for each of the 

* • 

' methods employed. 



Goal It . Tq compile" and/or de3ign instruments which, with minor al-* . 

terations, coulcl b'e used as a format 4n conducting* a ^ttldent tollow-u^ 

program', .* . . '\ " 

Objective 1: * Identify the various types of information to be 
* ft ^ 

' ^ • c , • 

obtained from respondents. ' , ' » 

Objective i: Establish a suitable mail questionnaire which could, 

with minor revisions, be made applicable .to^ other programs. 

Objective 3: Establish a. suitable interview schedule whiclY cojald, 

'A . • • • 

with minor revisions, be made applicable -to other program^. 

Objective A: Establish a suitable telephone survey which c<!^uld, 

\ with minor revisions, be made applicable to ather programs. . 

Goal III. .To incorporate and docunipnt tl^e methodologies most appropriate 

for carrying, out each survey technique. 

Objective 1: Extract from previous studies the most advantageous' 

method for conducting a mail questionnaire Survey. 

Objective 2: " Extract from previous studies the most advantageous 

.method for conducting a personal iptervifew survey. 

Objective 3: Extract from previous studies .the most advantageous 

method for conducting a telephone survey. 

^ Objective 4: Document the actual process of data collection re-' 

quired for each survey technique. ^ 

Goal IV: Ta obtain meaningful information from graduates and dropouts 

from the Skyline. Career Development Center that would provid^ a ba6is for 

program alteration or, at least, lend impetus to considering such changes 

(see the sample section for clarification) . 



All of these ^goals w^re part of a more ^ambitidns, and hopefully achievable, 
gqai~to increase the number of sophistica'ted and appropriately conducted 



|pllow-up studies of students by local schoal districts. 



Sample ^ • : 

^ Xhe SIcyline Career Development Center (CDC) , a school-bas^d career 
education program i\l9allas, Texas, was selected for the study bacauise of the 
availability of ;^eededfQata apd because the-student body included a large 
number of minority students. ' \/ ^ ^ 

All graduates ^(d^senioi's who did not return to th^ CD.C after t/he 1971-72 
school year were included in the study, the rationale for including all such 
persons was that the opinions *held and activities engagejl in by any student 
are worthvrtiile* and meaningful. Furthemfore, to ignore an individual simply^ 
because he failed to graduate wo\ildf be^a serious criticism of any follow-up 
study. 

After preliminary examination of this group, a total of 492 former 
sti/dents were. identified vho had completed at least one year, of study in the 
CDC and who were no longer, enrolled in the secondary schools pf Dallas. Nine 
potential subjects were dropped from the sample because of such reasons ^s 
* death, long-terta illness, and men,tal disorders. Table^J^hows the sex ^d \ 
. etHnic background o"f the sample. ^ , | 

All perfeons included in the population under study were sent a prelitninary 
questionnaire and a letter of explanation concerning the study in November, 1972. 
Approximately 40 per cent of the persons failed to respond; however, dat^ , 
were obtained on all but 27 of the subjects through relatives and. friends. 



■ The inability to .contact, aubjects via mail was due primarily to the following^ 
y ' * - ^ \ . ' ' ^ y . . .V * • ^ : * 

reasons:* students had left home^ ^ their families had move4, or information ^ 

in^the school's file was inaccurate as to the student ^address and/or telephone 

number. V^e 27 npn-resp(3ndents were kept in the 'study on the basis that they 

'might be. reached in^the more structured, treatments carried out ovjer a longer- 

« ■ » 

period of time, - , ' i 



Table 1 . 

Classification of ^Post-High School 
Population Under Study by 
Ethnic Backgrpund and Sex 



• 








Sex 










Male 




Peinale 




Total ^ 


- r 

Ethnic Bl^ckground 


n 


'Percent of 
Total 


n 


Percent of 
. Total • 


J n 


Percent of 

Total 

> 


Anglo 


171 




186 


52.1- 


357 ' 




Black . 


^ 23 


30.3 


53 


69.7 


76 


V '15.5 


Mexi can-Amer i can 


^ 21 • 


36.8 


36 


-63,2 


-57" 


11.6 . 


Other 


1 


■ 50.0 ' . 


■ 1 


. 50.0 


. -2 


. 0-3' 


Total . 


216 


, 43.9 


276 


' 56.1 • 


492 


100.0 ^ 



Assignment to Treatments 



The three methods of data collection ut\der study (hereafter referred to 
as treatments)' and 'the .maj or ' reason ' for their selection , were as follows: 

1. Mailed Questionnaire—selected be^cause it is the most widely re- 
searched"" and utilized method oi conducting, follow-up research. 

2. Personal Interview— ^selected because it should allow^ for ^ maximjim 

» ♦ / 

» contact between researcher and respondent. 



questionnaire treatment and Mexican-Amer leans in\he personal interview 



3. Telephone Interview— selected^'^ecayse it presumably combines 
desirable features of both the'maiXed questionnaire and personal interview , 
techniques. ■ • ' , 

The 492 subjects selected 'for the study were randomly assigned to the 
three treatment gro,ups. To ensure against biasing by assignment, no pre-"^ 
ferential assighmepts were made based W prior knowledge obtained^ from the 
data .collected in November^ 1972. Table ^3 Identifies the composition of ^ 
each treatment group based on. sex and ethnic, background/ The random assignment 
did place a slightly higher than proportionate ^umber of Blacks in the mailed 

t 

treatment ^roup. ' ^ ^ 

To examine whether or not substafntlve differences would occur based on 
treatment received y 48 subjects (16 from each treatment^ were randomly ^ 
selected to, receive a Second treatment. This selection was made, before any 
d^ta collection had. begun.' After tiie completion of data collection for all 
groups, attempts were made to obtain information utilizing th^ssecon4 treat- 
ment. These attempts were made even if no data wer^ obtained as aN;;esult of 
the fTrst treatment. By collecting the' data in this fashion, the/ orderN^f 
treatment was counterbalanced, thereby eliminating .the possibility of -biasing 
the results. 

Instrumentation 

•/ 

In an. effort to "enaure that the data obtained from eaTql) fcrftatment would 
be comparable, the content^ of the instruments for each treatment was Bi^t 
; equivalent. Only jninbr changes necessitated by the type of treatme^nt were 
'made,. . ^ • ' ' 




Content s The data* obtained from the instruments covered ^ach of the 
following:' ^ ' , ^'^ ^ ' ^ . V 

^ !• Background Data . 

rt'rthdate/ ' ^ " * •* 

i^^Pthnic ^ackgroand 4 X . " - 

. " Srea of?High Sdiool Speciallzatioti 
e; 'So6ial Security Number. • , • ' . . - 

2. Possible Status Changes r . ^ ^ 

a. Address ^ ~ ^ * ^ * 

b. PhoQe Number , ^ , ^' . 

c. Marltar Status' r ^ , 





3. Present -Employment Status 

^ a. Where Employed 
^b,VWhat Type of Work 
c,. f Duration of Employment 



4; The Pierception of )Foi;mer Studeijts with 'Regard * to CDC 
• ' - ' • ' # 

a. Strengths , 
. b. Areas ih Need of Improvement * > . 

General Level of^Satisf action • * 

e * 

5. Future isPlans ' ^ 

; ' ' > " ' ' \ ' ^ ^ » - , 

a. For Pers^Qhal Advancement . • 

b. Mobility^ . ^ , ^ V ' 

c. « Notif icatiqn of Futyre Stijdy ^ , ^ % » * . » 

Pilot Te^t, ^ To ascertain th6.^utility of the newly developed lnstrumen-6- 
.... * . - . ^ ' . 

tation, a pilot study was conducte4.' A sample of 'thirty twelfth-grade Skyline 

Center is'tudeats (ten fron^ach ethnic group) were selected across varlqus 

^caceer clusters. -In. the spring of 1973, Interview's were obtained^* from these 

stiral^nts utilizing the instrumentation. 

The foliWing, changes we?e indicated as a result of the findings of the ' 

pilot S6{i4y: ; , " ■. • ■ . • ■ ; . , ■ /- 

, - * ' ' . . • - ■■• . ' ? ' • 



HC ■ ■ . . ■ - . ■ 



1,3 



3. "Leading questions" that would .elicit responses that were desired on 
the parfbf the researcher." » 

4. Items th.at iStfmbined two or mare questions^ in a single f tem. » 



5. Questions tha^t might lead the respondent to-either refuse to respond 
or prevaricate in -his* |r her -response, * - ^. 

After the above changes had been incorporated, the proposed final forms , 

of instrumentation wepe reviewed by a group of Skyline seniors who felt that 

* ' *» 

the iristruments^did, in ^act> communicate; i.e., weire clear and unambiguous. 
.... Methodology *^ , ^^^--L^ 

* 

Since asrsignment to treatment occurred independent of such considerations 
as whether a respondent could be reached, by^ telephone or was located a 
great d^istance from Dallas, it was anticipated, that a number of potential 
respondents would be missed. However, random assignment to treatments was 
necessary to etisure that all treatments could be compared without biasing 
oytcomes. Future follow-up studies might wish"to take such considfeirations 
into account apd make appropriate alterations to improve respons^e rates. 

The- interview^, telephone calls, sthd mailings took place during the 

' • . , . 

months' o^f May' through August, 1973. To'ensure that each individual re- 

ceived similar treatment, oply four interviewers, each with previous interview 

experience, v.ere^used. The interviewers constantly interacted «7ith each other 

to analyze and ^tecommend methods to ensure^the highest jjuality of response 

■ • ^-^^ . .. ■ • ■ , 

and consistency interviewer technique. , A detailed.'^description of the 

procedures employe^ fot each treatment method^ follows . 



3. "Leading qiiesCions" that would .elicit .responses that were desired on 
the part" of the researcher • * 

' ^ * ' ' ^ ^ ^ 

4. Items that Combined two or more questions^ in a single f tern. ^ 

5. Questions th^t might lead the respondent to- either refuse to respond 
.or prevaricate in his |r her response. _ • - ^. 

After the above d)anges had been incorporated, the proposed final forms. 

of instrumentation we^re reviewed by a group of Skyline seniors who felt that 

- ^ { r ' ' 

tha instruments did, in i!act> communicate;, i.e. , weire clear and unambiguous. 

, Methodology *^ , ^ , ' ^ 

Since aarsignment to treatment occurred independent of such considerations 

r • ' ' * 

as whe4:her a respondent could be reached by^ telephone or was located a 

great distance from Dallas, it was anticipated, that a number of potential 

respondents would be missed. However, ratj^dom assignment to treatments was 

pecessaW to etlsure that all treatments could be compared without biasing 

outcomes. Future follow-up studies might wish"to take sUch considerations 

into account apd make appropriate alterations to improve respons^e rates. 

The interview^, telephone calls, ^d mailings took place during the 

months' ol May' through August, 1973. To' ensure that each individual re- 

ceived similat treatment, ofily four interviewers, each with previous interview 

* * 

experience, were^used. The interviewers constantly interacted ^ith each other 
to analyze and ifecommend methods to ensure^the highest jquality of response 
and consistency In interviewer technique.^ ^ A detailed^^'description of the 
procedures employed fot each treatment method^ follows . 



A 



* Analysis of tHe Data 

1 ; * y 



Once^the data were cdlle^cted for all three treatment. groups, tj^plcal 

" ' - J ' > ^ ^ ^ 

data preparation procedures were employed. The first- step required coding of 

If 

the information so that computer applications could be utilized \to categorize 
and analyze the data. • , " . s > i 

To minimize -the possibility of differences in interpretation of responses 

all coding was done by two individuals. Higher reliability was anticipated 

\ ' • 

by employing only two coders. To 'ensure validity in coding, a sample of 

completed instruments were^j^mitted to third parties for coding— the intent 

being*to see if a^co^set^lis as to the appropriate coding of open-ended 

questions.- This precaution, however, did .not replace the normaj^eries C>f 

' random checks' to ensure interrater reliability; 



\ 



All coded data were then keypunched and verified for computer use, 
- After initial computer runs, which iti%ud^l^tople cross-tabulations, it wa^ 
necessary to correct some erroneous coding and to adjust fov the large 
number .of responses codeji as ''other" due to the channeling' of responses to 
open-ended questions' into^ eight possible response cat^^SrTe^. While some. . 
specificity was lost^at^this stage^ *the gains in utilization of findings 
warranted limiting the possible -response categories. / 

The comiJuter analysis of the data involved, the establishment of a 
series of crosa-tabulations based On sex, ethnicity, durriculum background, 
and treatment. This was done to prepare the responses foiT data analysis 
*.tc^ilizing the c^ii-fequare test of goodness of fit foir fndepei^eiat samples. 
The chi-square test was chos'en as the most suitable test ^for atiQlyziilg the 
data since the data were in frequencies and the measurement (^] the variables . 
included those in a- nominal .s^le and in discrete categoAes Of an. ordinal 
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scalers All other tests commonly used for tests of independent sample^ require 
aKleast otdinal measurement gf th^ variables; hence, the chi-square test is 
> uniqMely useful for ^lata such as that collected for this research. 

In essence, the chi-square test 9as used to ascertain if the samAles 
had come from the same population or from identi^cal populati6ns with^ r^pspect 
^ to the pi:oportion'of cases in the various categories involving response 
ra,tes for the various treatments effectiveness of these treatments with 
different ethnic groups, and nature of responses obtained. 

(Questions concemiiig cost and cost-effectiveness were examined by • 
totaling , costs and dividing by the number of responses. Finally, the 
analysis of the most advantageous design for future studies involved^the ^ 
weighing of information obtained ia ex^pining previous question^.- ^ %^ . 

Findings 

I 

Question One ^ - f 

The first question 'addressed was: Which method of conducting a follow- 
up study elicited the greatest percentage of response?,- 

' Attempts were made to follow-up a total of 492 former students of 'the % , 
Skyline Career Development Center. _ After pursuing the methods outlined 
previously, 269 individuals finally responded to the survey. This gaye 'an 
overall response rate of 54^%., The null hypotjiesisi that the proportion of 
^ responses was- the same for the three modes 'of folJ.ow-up was tested, and a 
statistically significant difference (% =12.71, df = 2-, p < .005) was • 
found among treatments.-* Examination of the data indicated that the mailed 
questionnaire and personal interview approaches were approximatley equal In 
effectiveness eliciting response rates of about '50 percent* The telephone 
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survey method was consideijabfy more effective than the other two methods with 
a response rate of approximately 66 per cent. Further analyses of the data 
comparing> the frequencies '^fir e&ch possible pair of treatments indicated thats 
the telephone survey tech'nique was significantly mor-e- effective- than either 

* 

the mailed questionnaire or personal interview methods J while no significant 
difference was found between the latter two treatments. 

The reasons for .not receiving responses from subjects varied" among 
methods. The most .important Reasons are listed in Table 2. with the number 
of non-respondents for each reason shoWh by treatment. Obviod^ly, the|e 
reasons are not necessarily m'utually exclusive, since some potential' re- N 
spondents who were never contacted might also have, ended up in the. "refuse 
to respond" category*. 



Table 2 

Comparison of Occurrencfe Among Treatments 
with Reapect to Reasons for 
not Receiving Responses 



Reason 



Mailed 
Questionnaire 



Personal 
Interview 



Telephone 
Survey 



Total 



No telephone 

' No forwarding 
information' 

/Wrong number or 
^ . disconnected 
^telephone 

.^Out of Dallas area 

iould not follow up 

Refused to respond 

Tptal' number of 
' non-respondents 



1 

0 



11 



14 



12 ' . 


' 5 


11 


28 


5 


13 


19 


37 


24 


30 


14 * 


68 


3 


8 


• 1 


12 


81 


86 . 


' 56 


223 

t 
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Inspegt^ion of the data displayed ±n Table 2 reveals that methods re- 
quiring some form of personal contact with the subject were less "effective 
when dealing with subjects who had left the immediate area; however, this was* 

to be e^^ected since no long-distance. calls were. Other interesting findings 

/ \ ' ' - • . 
showed that: 

/ 

|l. There were appreciably fewer respondents who could not be fojllowed 

up via the telephone as opposed Eo the other. two methods* 

/ 

/ 2. Respondents were most likely to refuse to respond when asked for a 



personal interview, , , ' 

I " 

/ 3» Quality of^ data (i.e., phone numbers and addresses), inability to 

follow up (i.e., line busy, no answer, etc.),, and respondents living outside 

^ ■ -x 

the" Dallas area wer^' the major reasons for non-response. ^ 

S Question Two 

The, second question posed was: Were there any differences in the nature 

of responses elicited by each method of conducting a follow-yp study? 

^ - ' ' 1 

This question was posed ta examine two aspects of responses. The first 
dealt with communication; that is, the extent to which respondents gave no 

- . i ' 

response to various questions. The second aspect examined whether or not 
personal contact, either by telephone or in person^ would inhibit respondents 
thereby lessening t:he^> chance that a respondent would comment negatively' on 
certain subjects. The randomization of subjects to treatment eliminated 
potential bias in favor of any onejnethod of seeking response, thereby 
allowing for examination of differences in the nature of response. 

The extent to which respondents were able to formulate answers to 
questions, as opposed ^to failing to respond', varied significantly from one ' 



Treatment to another. Answers to questions concerning satisfaction W^ith 
instruction and counseling, and open-ended questions dealing with the* Skyline 
Center, sliowed that former students responding, to a mailed questionnaire had 
a much higher frequency of no response, especially for questions cottd^fning 
the CenteiT.^" . ' * 

Two questions were identified as suitable for assessing the willingness 
of respondents to make negative comments. These questions involved the * 
rating of instruction and counseling tti a five-part scale ranging from 
excellent to poor. ^ * , • 

# 

There were significant differences in the number of negative responses 

received depending on treatment and the question being asked. Evaluation of 

instruments showed that the only statistically, significant difference 

(^2 „ 4,83, df = 1, p < .05) on a paired comparison basis existed between 

the mailed questionnaire and personal interview approaches. *No statistically 

significant results were obtained when comparing the mailed questionnaii^^ 

. - : ' ft. 

against the combined efforts of the other two methods. The rating of! counseling 

differed In that significance {% = 11.48, df - 1, p < .005) was f<»nd 
between the mailed questionnaire and the telephone interview. Howevd^, the . 
mailed questionnaire had a statistically .significant differenc^in the nu^ibet 
of negative, responses when compared to the combination of personal forms of 
data collection. Therefore, while the mailed questionnaire did ^elicit a' 
greater number -of negative responses than personal forms of data collection, 
the lack of any consistency across the two questions, particularly when 
comparing the mailed questionnaire and the telephone survey results , Indi- 
cates that further research is needed with ragard to the effect of personal 
involvement on willingness ^to respond in a negative mode. ^ 
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- ' Question Three » - ' - * 

i 

-. , ^ The third que^ion dealt with jcosts: What were the comparative costs 

for each' method? • • ' ^ 

( - ■ ' ■ 

Fixed (fc^^ . ' '/ . \ " 

In examining' this question, it is important to recognize that there 

^ere fixed .costs ^ such as printing ar^ processing', that applied commonly to 

all three treatjnents. .While these co^ts did not vary among treatments, their 

eaeistencq must , be ^considered in cost estimates. The fixed costs, however, 
- ' , ' • * , 

. were relatively small since "the Pistrict provided printing and computer time 

' to the , investigator at cosj:. Total" costs for printing and processing were ^ 
approximifeely 20 dollars -for printing and eight dollars for comp^ut^r time.^ 
^ Manhour costs can be ^>roken '<iown* Into; three categories: (1) px^or to 
' treatflient, (2) duriitg treatment,, atid (3) after treatment. Each category will 
> ^ be examined separately tecau^e time p an 'bfe saved in future implementations 
^ since certain developmental stages used need not be repeated. 

Furthermore, the. types of ^manpower required varied greatly depending on 
' the task at. hand, Fot ease of providing meaningful comparisons of manpower 
costs, clerical level co^ts were calculated at three dollars per hour, re- 
search assistant costs were fixed at Jive dollars per hour, and senior^ 
researcher costs were set at,$«n dollars per hour, 

I ^^^^ ^ " ^ *^ • 

F'pior'^to treatment, the research-'^taf f was^tequired to construct an ^ 
instrumenty- randomly* assign former s^ude^ts to treatments, ^nd plan a 
design. Approximately 60 hours of .work (including pilot te^stin^^were re-^^ . 
^ quired. This included 50 hours pf senior researcher time, five hours of re- 
search assistant work, and five hours- of clerical help. In the future, 

h * • 

20' 



this stage should require less than eight hours 'since instrumentation has 
been completed. , - > , ' 

During treatment, monitoring of staff activities required a great deal ^ 
of time- on the part ^f the investigator. It is impossible to estimate •• _ 
precise^ly the total nSeT of hours spent in this activity. However, the total 
amount of time was extensive, .involving a portion of each day, during. the time, 
data were being collected. This was considered as a role respon^bility of 
the senior researcher involved in the study. , , ^ ^ 

. The greatest amount of required manpower was needed after the collection 
of. data. was completed. This entailed the coding, keypunching, verifying, 
and analysis of the data collected. The time required for these tasks were: 

1. approximately 57 hours of clerical assistance fpr coding, 

2. approximately 10 hours of clerical assistance' for keypunching, 

3. approximately 40 hours of clerical assistance for verification^ and 
A. approximately 60 hours of senior researcljer time for analysis. 

Verification and analysis of the data were greatly facilitated through the 
use of electronic' data processing. 'It would bp safe to assume that if a 
computer were not available, the time required for manipulation o.f data by 
hand would Inhibit much of the analysis necessary for meaningful interpreta- 
tion. . • ^ 

Treatment Costs * ^ , . 

Costs peculiar: tp treatment varied* greatly, depending on the treatment 
involved and particularly the methodology employed. > 

The initial* response to the mailed questionnaire was 37.8 Rer cent. . 
After one telephone reminder, tW response rape , increased to 42.7 per cent. 
A second telephone reminder, brought an additional 7.3 per cent , response. 



A third call was made to one student' who subsequently sent his questionnaire 
back. This- process obtained 83 responses out of a possible 164, or a 50.6 
percent response rate. 

A total of 164 questionnaires were sent out initially at a cost of. 
16^' cents each (this cost includfed the self-addressed, stamped return 
envelopes). Seventeen additional questionnaires, of which nine were returned 
completed, were sent upon request at a cost of eight cents each. Thus, 
•twenty-seven dollars and sixty, cents was. expended for the mailing of question- 
naires. * ' 
\ ' As far as manhour expenditures were concerned, 167 telephone calls were 
placed to encourage response. This procedure was employed as an alternative 
■ to repeated mailings which require longer time frames and for which there Is^ 
documented research "demonstrating limited success. The average time required 
per call—this included clarifying telephone numbers, busy signals, no 
responses, etc.— as recorded on data sheets was approximately 12 minutes. 
Based on this finding, 33 hours and 24 minutes of clerical services weire 
expended by research personnel on follow-up of questionnaires. This means 
that 24 minutes of time were required to obtain a completed questionnaire,. 
This finding should dispel any belief that fewer human resources are required 
in a mailed questionnah© unless one^ con'siders .repeated maili-ngs. Manhour 
.expenditures not prev'iously^mentioned included labeling and stuffing envelopes 
checking returned questionnaii^K and assimilating information obtained 
'through telepT»o'ne follow-up dat^. These totaled approximately 12 clerical 
manhours. 

The telephone survey, as conducted in thl^v^udy,- required no immediate 
out-of-pocket outlay. .The manpowet expenditure wasN^ involved, hence 
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separate time analyses ,were prepared for non- respondents and respondents 
before combining the two analyses to obtain the total cost for this treatment. 

Three hundred and three calls were' made in obtaining the responses of 
108 former students. The actual interviews taken over the telephone required • 
approximately 15 minutes each for a total of 27 hours of research assistant 
time. The" preliminary calls to reach the 108 respondents (65. a per-cent' 
.response rate) averaged four minutes each for a total of .13, hours of research 
assistant time. Hence, the actual average number of minutes required to 
obtain a response from the group of respondents was 22 minutes. 

One hundred and sixty-four telephone calls were placed to non-respondet^ts 
and in only 27 cases was it found to be idpossible to reach the subject via ' 
telephone. The average time spent on each call, as recorded on cover sheets, 
was six minutes. Therefore, a total of 15 hours and 12 mVnutes of research 
assistant time was expended on attempts to obtain an interview from 56 former 
students. Cotnbining the above manpower data shows that 55 houts and 12 minutes 
at five 'dollars per hour were devoted to this method of obtaining follow-up^ 
data for a total cost of two-hundred seventy-six dollars. 

The personal interview 'technique was similar to the telephone approach 
in that there existed no out-of-pocket costs, to the investigator. However, 
forv^ purposes of making comparisons, interviewer ^'mileage ^Was converted- to 
doMa4 figures at a rate of ten cents per mile. In addition, there were 
several other factors such as location and time of interview, preparation for 
interview,, and clerical organization that required special a-ttention. ) 

Sixty per cent of all interviews were conducted^ at the Skyline ■Cenfer_ 
during normal (8:00 A.M.. to' 5:00 P.M.) working hours. Twentjy-eight iaterviews 
were completed at the respondents' homes at varying times with the majority 
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occurring in the evening. *The ^verkge Siistancfe from Skyline was" seven miles 
one-way. However, by scheduling and Obtaining interviews by area, mileage' 
was kept to 'minimum. Thus, 284. miles, were logged in conducting interviews 

at respondents' homes. Three* additional trips, totaling 18 miles ,^ yere made 

> • • t 

' \ ^ ^ - 

to obtain interviews at. the subjects' place of employment* Monetary costs 
travel, on the basis of ten cients per mile, came to thirty dollars and 
twenty cents. Tr-avel time for interviewers totaled 12 h hours. It is also ^ ^ 
noteworthy that while appointments made with some individuals^ for interviews 
at Skyline were fnissed, no trips to 'homes or places of employi^ent proved 
fruitless. ^ , ^ f . *' 

' ,The average time reqirfred for the interview itself was 20 minutes. 
However, 495 telephone calls were placed, 93^ appointments were made, and 
approximately 12 hours of clerical time was spent in planning and as§imi3feting^ 
various pieces of iri^oriaation reflated to the personal* interviews. 

Telephone Calls averaged eight minutes each, including time spent in^ 
looking up changed numbers , ^etc. ^Therefpre, a total of 116 h manhours, 

^ / 
1. * 

' 38 J$ by research assistants and 78^ by' clerical personnel, was required for 
the personal interviews at a' cost of fojur-hundred' twenty-six dollars 'and * 
fifty-cents. ^ ' ' ^ ' . - ,^ 

D^^sregarding those costs that app],ied equally td' all three ^eatmentSy ' ^ 
it was found that (1) the mailed questionnaire required approximate^J.y 46 ♦ 
manhours and twenty-seven dollars and sixt/ cents, ,for a total cost of , , 
one-hundred sixty -five dollars and sixty cents;, (2) the . telephone "survey ^ 

' required approximately 55 loanhours and nq out-of-pocket expenditures ^ 

(this is, of course, siibject to local telephone rates ^and the availability, 
\ - * * * . 

of telephones for interviewing purposes), totaling two-hundred seventy-six 

/ 
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dollars; and (3) the personal interview method required *^ppro:^ately 116 
manhours and thinty dollars and twenty c^nt^for a^total cost^f fdu^undred 
fifty^^x dollars and sev^nt^ cents^^ 




Question, Tour 

T|ie fourth qu^tion investigated was: Which method.! 
methods was most cos t-efi^tive for conducting a large-sc 
stydy? ^ ^ % ' V 

TKfere are ways of examinin^^oost-ef fectiVeness. ^C|ie more simple 
^approach is to simply compare .rate of ^e^ponse with^^al cost^<^an individual^ 




treatment basis. A more <^mplex method of' assessing cost-effectiveness is to 



weigh the nature or quality oY^esponse in conj,uh<;t^^ with rat^ of res^otise 



and arbitrarily det'ferinine ^this valb^^gainst cpst. 

^ When comparing cost with r^te of fe^^onse, , the follo'^i^g ranking of 

cosfi^ef fectiveness- was obtained: 

\ 

,1. The mailed questionnaire with subs^quenKfoHo^-up requir^a^6 
^ahhours ^n^d^wenty^seven dollars and sixty cfents, at^ total ^ost of 
one-hun^^ed sixty-five dollars and sixty cents, to obtain ^3^responses out 
of a possible 164". X This, meant that on a per response basis, fcBe^cost was 
app](pximately twbvdollars. ' flad no follow-up been eiaployed, ^he>^^. would 

vtwo cents per resoctosey^ Th\is». the re^qnse 
rate would have beW pnly\^8 per cent but the cost-effectiveness was consi 
erably hi^h^ _ ^ V \J \^ . - \ 
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2. The|telephone survey had\;he highesV^ate of response, l^^ut o^ 
- 164, and a total cost of two-hundred ahd seventy^^x^lla^s. The 

per response basis was two dollars and fif^five cen^s^wi^no 'other ex- 



clusive treatment cost^.- 
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The petsonal i^iterview approach had a combined cost/bf four-hundred 
fifty-six dollars ancC seventy centj, including manpower and out-of-ppcket 
costs, to obtain .tesponses » f rom 78 of 164 individuals. On a per response^ 
ba^is, this meant that five dollars and eighty-six cents, was expended^ 
Hencfe, this method wap obviously the -lea^t. cost-effective wh^n one examines 
only: r^te of .responae, ^ » * , * 

rrhe inplusion of nature or quali-ty of response into a com^>arison of ^ 

• * . ^ ' • * , / 

cost-effectiveness requires that' value judg;ments. be made regarding l^ck of . 

response, "extent of response, and willingness of the respondent to express. 

, critj^isms^. , .Each of these issues must be weighed in sucH a way' that inferences 

regarding eJEf ectiveness may be drawn* It was the opinion, of the researcher*, 

that obtaining a response and the ^subsequent validity of that response were 

most important. This flatter concern dealt with the respondents' willingness^ 

to express cri^ticism. The extent or dimension of the response (such as numbei^ 

of words or thoughts) was considered tc be- of relatively less importance.- • 

Based on the above considelE'ations , in conjunction with rate of response,- 

the following interpretation of cost-effectiveness appears warranted. The 

telephone survey consistently had substantially fewer non-responses than the 

mailed questionnaire witfi no lack of willingness to express criticism when 

responding; hence, the telephone survey would be considered the m.ost cost- 

effective particularly when one als.o considers the difference in rate of 

response as compared ro the .mailed Questionnaire. The low response rate far 

the personal interview, combined with noticeably fewer critical comments 

'indicated that this method was the least cost-effective. The arbitrary 

nature of^adjusting on the basis of one*8 personal interpretation of the 

importance and weighting of quantity versus quality of responses may allow 



other rankings- of cost-effectiveness; however, the ab<3ve interpretation appeared* 
quite reasonable on the basis of the data obtained. ^ ^ 

^ * Question Five 

The fifth question addressed was:' Were some! >me"thods more effective • 
^^'ftjian others for certain ethnic groups? 

It was esfpected that the three treatments would elicit significantly 
different response fates 'from respond-ents who were members of minority . 
groups. There did exist a statistically significant' difference among the 
treatments, when considering all students. Based on this finding, comparisons 
were made of the three treatments for each -ethnic group* ' . 

A statistically significant difference was found among ethnic groups, ^ 
Additional examination revealed that «the difference could be attributed to 
the higher response rate obtained from Anglos. 

Comparisons iy pairs of ethnic groups shpwed no difference between Blacks 
and Mexican-Americans wheTi consolidating respondents across all treatinents. 

. Findings concerning fate of response by ethnicity -showed that all 
methods of survey research employed in the study generally were more .^fective 
with Anglos than with Blacks or MexicanrAmericans, HoveveT, the response 
from Anglos td the telephone survey method was statistically significant * 
(^2 ^ 8,-64, .df « 1, p < .005) vhen compared to the personal iti'terview approach 
and was ^ contributing factor to the dif-f^ences identified, _ - /^-^ 

While no significant differences were obtained -in tjie ex^ination of ^ 
treatment comparisons for Blocks and Mexican-Americans, .the small gample^^ 
size, particularly for Mexican-Ame;ricans^ made it difficult to obtain ^ ^ 
. statistically significant differences. ^The observable differenced may*only 
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be cKance occurrences > but the response to telephone contact was very en- 
couragin^g", especially when one considers . the poor response rate to the mailed 
questionnaire • ' _ , . . , ^i*^- 

It was apparent thar' the,, telephone survey elicited a higher response 
rate for Anglos and,^ while' it was not significantly better for Blacks ^.and 
Mexi:^an-;Americans from a statistical standpoint , *the direction of differences 
favored. th^ telepJhone survey. ' ' , ' -^f- 

* V 
\ ' * ^ ' 

Discussion an<J Interpretation , 

* 3 

The conclusions discussed in this section wete based upon findings re^ 
suiting trom the Ijuestions posed and must take , in to account the limitations 
and procedural Considerations of the study. ... 

No Substantive differences Based on Treatment ' ^ , / 

^ Of the 48 subjects' randomly selected to receive a second treatment, 19 
did in fact respond to two methods of thfe follow-up survey. It Is 
noteworthy that as with the study in general the telephone^ interview^ tech- 
nique had the greatest number of respondents for the second treatment, 
was also found that^ fpur subjects who had failed to respond to 'the initial 
treatment did respond to the second treatment. This Iticluded tfeTree indi- 
viduals who could not be reached by telephone or in person due to either, 
location or lack^f tfel'ephone^ service* that returned the mailed question, and 
one former student who had failed to return the mailed questionnaire but was 
interviewed over the telephone. ' * f ^ 1 

As far as substantive differences due to the treatment received, there 
Was no consistency in the responses obtained to support any arguement that 
the same' individual would respond differently to the same^question depending 



on method employed, to 'obtain said response. Certainly the smal^ sample size 
makes it difficult to unequivocally state that no difference should occur; 

* * y , • 

.^howev^^x^ no indication of differences was realized, ^ ' \ 

' ■ — . ■ f - 

Differences In the Level of Item Non-resp'^nse^gcrossTreatments 
Thete was no support 'for the Hochstim and AthanasopouTd^- 

finding concerning the completeness of questionnaires being equ^ acrpss 

5 

all treatments. There w^s a statistically significant decrease?^n ::he number 
of subjects choosing'not to respond to various questions when pirsonalized 
methods of follow-up were administered. It is probable that tli.^ subjects of 
the earlier study were not representative of former high school students from 
a large urban school district such as Dallas. ' 

' Comparability of Findings across Treatments 

Consistent with' the findings' of Hochstim and Athanasopoulos (1970) 

.but 'in opposition to those of Jackson and Rothney (1961) results'of the 
.present study suggest that once responses are obtained via any of the 
strategies there is no reason to expect more complete. .or illumijlatin^ 
responses because of personal contact or anonymity of respondents. Closer 
ended questions were found to elicit almost identical response? except in 
the case of two ratings involving the quality' of instruction ^«nd counseling. 
These instances demonstrated that "^he mailed questionnaire do|s obtain a 
greater proportion of' negative responses than at least one ofathe personalize 
methods; hpweveir, there was jjj^ consistency in the findings of: the present 
study to' support the mailed questionnaire over either the t^^ephone or 
personal interview techniques.' On open-ended questions for-^hich responses 

were obtained, ,the results demonstrate that there is, no rea^ti'to expect 

' < 

appreciably different responses for any of the treatments. -I " " 
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Differences in Rate Response across treatments . 

Related studies had produced findings which indicated that the rate o^ 
response should not differ significantly for any of the follow-up strategies 
, employed. 'The present study fo.und this to be trtie for the mailed quest i<}nnaire 
and personal interview^ .however, the telephone interview technique did obtain 

a si-gnificantly higher^ response rate particularly for Anglos when compared 

■v * ' 

with the personal interview. While statis^:ical significance was not found ' ^ 
in the differences among treatments for Blacks and Mexican-Americans, this 

^ may be attributed to the small number of minority subjects who actually 
responded. The lack of minority representation with regard to interviewers 
(both telephone and personal) may have also contributed to this lack of 

* statistical significance when comparing personalized methods with the mailed 
questionnaire. Across all ethnic groups the results of the present study 
sapport increased awareness of the value of telephone . interviewing for 
achieving higher response rates. 



Differeoces in Cost across Treatments 

The cost analysis^ findings of the present study are consistent with 
previously reported research (Hochstim and Athanasopoulos, 1970; Willardsen,^ 
1972; Orr and Neyman, 1965; and Jackson, and Rothney, 1961).. The 'personal | 
interview technique is ^over twice as expensive as either the mailed question- 
naire or telephone interview. The costs for. the latter two were found to be 
quite similar wheo^^c^pared on the basis on the number of responses obtained 
by each. The other cost aspect that was important to understand is that much 
of the cost incurred in follow-up researcH^involves human ^resources. This is 
particularly important when one considers the type of individuals required 



1. 



. . > . - 

'a 30 . . , 

31 



for various tasks. Th& mailed questionnaire except for analysis can be 
conducted almost solely by clerical personnel once instrumentation has been 
developed. Similarly telephone interviewing can be undertaken by clerical 
personnel who have been given training as to appropriate procedures. Personal 
interviewing does seem to require specialized skills that are most fre- 
quently found in college" graduates. 

Therefore cost analyses are deceiving unless consideration is paid to 
Vhat kind of costs are being incQrred such as capital outlay versus human 
costs. This is. combined with the above mentioned variety'-'of costs tied to 
human resources which can drastically affeet overall costs of the survey. 
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Implications for future Follow-up Surveys 

Considering the. need for constructively designed follow-up research to 
provide information for improving existing programs and formulating new 
programs, the ultimate goal of the present study /was to make meaningful 
suggestions for insuring the highest quality of resultant data. 

^or large school districts "^id programs with adequate staffing patterns, 
the conduct of a follow-up survey may have greater flexibility than small 
districts' whose need for such evaluation is no less. The results of the 
present study supported by related research indicates that more attention 
should be paid^o telephone interviewing, a technique suitable for both 
large and small districts. The limitations involviVig manpower requirements 
and locality of the potential respondents can be counteracted by incorporating 
mailed questionnaires into a study for those subjects unavailable by 'telephone 

• Procedures to incorporate in any follow-up study to insure the best 
possible results should Include the following: < 
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1. Careful identification of the population 'to be studied paying 
particular attention to otfain accurate addresses and telephone numbers • 
Frequently follow-up studic^ are not adequately planned so that there is 
difficulty in locating former students, 

2. Greater use of clerical personnel whenever and wherever feasible 
withou^.3^crificing quality. This wdul& require taking necessary steps to 
provide training and leadership for such individuals to insure the quality of 
their workr There is no need to use higfi salaried individuals for tasks 
perfectly suitable for more cost-effective personnel. ♦ 

_3, Instrumentation that is goaf lyriented so that results of the study 
may be tied to program improvement policies. Traditionally follow-up^ 
studies only seek answers to simplistic questions having little potential 
influence for school improvement regardless of the findings. Once subjects 
are contacted, it costs no more to ask in depth evaluative questions. 

In conclusion, the study conducted here, indicates that a carefully 
conducted telephone interview survey performed largely by clerical personnel 
combined with mailed questionnaires to subjects living outside the immediate 
community or those not having telephones sliould insure a response rate of at 



least 70 per cent. Fuj^ermore, the nature of the information obtained via 
this process should W equlv,alent 'to the much costlier data achievable from 



a personal interview approach. 
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